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TITLE: "PANORAMIC SURVEILLANCE SYSTEM" 

Technical Field 

This invention concerns surveillance systems. More 
particularly, it concerns imaging systems having a single, 
5 stationary, camera, for use in the panoramic surveillance of 
a space. 

Background to the Invention 

With recent increases in the occurrence of the crimes of 
theft and armed robbery, it has become commonplace to 

10 install a system for the surveillance of, supermarkets, shop 
checkout areas, banks and other spaces. Conventional 
surveillance systems use either multiple video cameras (each 
monitoring a zone of the space) or a -single, mechanically 
scanned camera. The former of these systems requires the 

15 observer to check several image displays to monitor the 
entire field of view. The latter type of system displays 
only a narrow field of view at any instant, which reduces 
the value of the system as a surveillance tool. 

Both of these conventional surveillance systems are quite 
20 expensive, requiring either multiple cameras and monitors or 
a mechanical drive for scanning the camera. Systems 
employing moving cameras consume more power and require more 
maintenance. A further disadvantage of the moving camera 
system is that any motion of an object that is within the 
25 image is difficult to detect from simple observation of a 
monitor screen, due to the apparent continual full-field 
image motion that is induced by the movement of -the camera. 
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^ Disclosure of the Present Invention 

It is an object of the present invention to provide an 
imaging system for use in a surveillance system, which 
utilises a single camera, which requires no movement of that 
5 camera, and yet which has a panoramic field of view that can 
be displayed oh a single monitor screen. 

This objective is achieved using a suitable camera ( an 
electronic camera - either a "still" camera or a video 
camera - having a charge coupled device (CCD) sensor at its 

10 image plane, being particularly suitable) and a dome-like 
convex mirror. Typically the dome-like convex mirror is 
mounted on the ceiling of a room to be monitored ( though the 
mirror may be mounted in other locations whenever it is more 
appropriate to do so in view of the nature of the 

15 surveillance to be performed). The camera is mounted 
relative to the mirror in such a manner that a major part 
(if not all) of the curved surface of the mirror is within 
the field of view of the camera* The mirror has a profile 
such that it projects most - preferably all- - of the 

20 environment in which the mirror is mounted onto the imaging 
plane of the camera - 

Thus, according to the present invention there is provided 
an imaging system for a space, said system comprising (i) a 
camera and ( ii ) a mirror, said system being characterised in 
25 that: 

(a) the mirror is a dome-like convex mirror, having a 
generally smoothly curved surface; 

(b) the camera is mounted relative to the mirror so that 
at least a major portion of the surface of the 

30 mirror is within the field of view of the camera; 

and 
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<c) the mirror has a profile such that the mirror 
projects radiation from at least a major part of 
said space onto the imaging plane of the camera. 

Except when special surveillance requirements dictate the 
5 use of a mirror of a different type, the dome-like mirror 
will be a circular ly~ symmetrical mirror, with no sharp 
discontinuities of the profile, and the optical axis of 
the camera* s field view (the optical axis of the camera) 
will be aligned with the axis of symmetry of the mirror 
10 profile. 

Preferably an electronic N camera having a charge coupled 
device (CCD) sensor at its imaging plane is used in 
implementations of the present invention. Such a camera 
(which may- be a video camera or a "still" camera which 
15 produces an image in a fairly long time, of up to one 
minute) will produce a warped image of the space being 
observed, which may be displayed directly on a monitor 
screen. Alternatively , the signal processing of the 
output, of the camera sensor may be modified so that the 
image co-ordinates are changed and a de-warped image is 
displayed on the monitor screen. If movement detecting 
software is included as part of the signal processing 
package, any movement within the region being monitored 
will be detected and the attention of an observer drawn 
to it (for example, by displaying the image of the zone 
where movement is occurring in a colour which contrasts 
with the colour of the remainder of the image, and 
preferably also by generation of an audio tone). 
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The significance of these and other r optional- features of 
the present invention will become more apparent from the 
following description of embodiments of the invention, 
which is provided by way of example only. In the 
. 5 following description, reference will be made to the 
accompanying drawings. 

Brief description of the accompanying drawings 

Figure 1 shows one mirror and camera configuration that 

may be used in implememtations of the present invention. 

10 Figure 2 illustrates the polar and cartesian co-ordinate 
systems used in this specification when defining the 
profile of a mirror. 

Figure 3 contains the profiles of mirrogs^ having 
elevation gains (defined below) of 1, 3, 5 and 7, when 
15 the mirror and camera configuration shown in Figure 1 is ; 
adopted . 

Figure 4 shows a compact form of the mirror and camera 
configuration, which utilises a supplementary mirror to 
establish the required panoramic surveillance. 

20 Figure 5 is the monitor screen image of a sguare room, 
2.5 metres high and with each wall 5 metres long, 
obtained using the mirror and camera configuration that 
is shown in Figure 1. 
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Figure 6 is the monitor screen image that is obtained 
25 when the image of Figure 5 is de-warped. 



WO 95/06303 



PCT/AU94/00501 



- 5 - 

Detailed description of the illustrated embodiments 

In the arrangement shown in Figure 1, a dome- like convex 
mirror 10, which has a circularly- symmetrical surface, is 
attached to the ceiling 11 of a room that is the subject 
5 of surveillance. This is expected to be the most usual 
way in which the surveillance system will be used but, as 
noted above, this invention may be- used in a variety of 
monitoring situations, including situations where it 
would be inappropriate - or impossible - to mount a dome- 

10 like convex mirror on a ceiling, a high beam or a high 
platform. The profile of the mirror 10 may be any one of 
a large number of practical profiles, several of which 
are described in more detail below. The mirror profile 
w ill he selected according to the magnification 

15 requirements of the surveillance system. 

A camera 12 is mounted directly below the mirror 10, 
facing upwards. That is, the camera is mounted with the 
axis of its field of view collinear with the axis of 
. symmetry of the mirror profile. With this arrangement, 

20 the mirror 10 will project radiation from a large portion 
of the room underneath the ceiling 11 onto a circular 
region of the sensor at the image plane of the camera 12. 
Thus the image produced by the camera (a typical image 
being shown in Figure 5 of the accompanying drawings ) 

25 represents 360° of the monitored environment in azimuth 
and from" -90° (that is, vertically downwards in the 
arrangement shown in Figure 1) to positive values (that 
is, to above the horizontal ) in elevation. in practice, 
the lowest elevation is determined by the size of the 

30 camera and its mount, which blocks the view directly 
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downwards from the central point of the mirror 10, and 
the highest elevation is determined by the axial length 
o'f the mirror and the shape of its profile. 

In the circular image that is generated by the mirror 10, 
5 a given radial direction corresponds to a specific 
azimuth in the environment, and Increasing radii 
correspond to increasing elevations. The profile of the 
mirror is preferably chosen to map equal changes of 
elevation on to equal changes of radius in the image. 
10 The elevation gain a is defined as the increase of the 
angle of elevation (Ap) per unit increase in the angle of 
divergence from the optical axis of the corresponding ray 
(Ay) entering the camera through the nodal point, as 
illustrated in Figure 1 . 

15 The profile of the circularly symmetrical mirror 10 can 
be defined in polar or cartesian co-ordinates, according 
to any selected co-ordinate system. In this 

specification, the polar co-ordinates . (r,6) and the 
corresponding cartesian co-ordinates ( x , y ) for the mirror 

20 are those shown in Figure 2, where the point {0,0) in 
each system corresponds to the point where the aperture 

y 

lens of the video camera 12 is positioned. 

The profile of the mirror 10 can be adjusted to obtain 
any desired value of elevation gain that is greater than 
25 unity. It can be shown that for any elevation gain a, 
the requisite profile in polar co-ordinates must satisfy 
the equation: 
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Sin[A+0.5 (1+0)6] =C.r. [-0.5 (l+o) ] 

where the origin . Is at the nodal point (the aperture lens 
of the camera), r is the distance to the mirror profile 
from the origin along a direction inclined at an angle e 
with respect to the optical axis of the camera (see 
Figure 2 ) ; C is a constant whose value depends upon the 
nearest distance of the profile to the nodal point, and A 
specifies the inclination of the profile to the optical 
axis of the camera at the point on the profile which is 
nearest. 



Examples of mirror profiles for* a few selected values of 
elevation gains are given below. in each of these 
Examples it has been assumed that A=9G°(that is, that the 
surface of the mirror is normal to the optical axis of 
the camera where the axis of the camera intersects the 
15 mirror 10). 

Example 1 : Elevation gain « 1 

Although a mirror having an elevation gain of unity would 
not be used, in the present invention, this example is 
included for the sake of completeness of this 
20 description. The polar equation specifying the profile 
of the mirror is 



cosB 



where r G is the shortest distance between the profile and 
the nodal point. m cartesian co-ordinates, the 
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corresponding equation (with . reference to the same 
origin ) is 

x=r Q , Vy 

where x denotes axial distance and y denotes radial 
distance, as illustrated in Figure 3. These equations 
5 define a plane mirror with its plane normal to the 
optical axis of the camera, at a- distance r c from the 
camera's nodal point. (Values of A other than 90° yield 
cones of semi -vertical angle A. ) 
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Example 2: Elevation gain » 3 

The polar equation specifying . the profile of a mirror 
having an elevation gain of 3 is 



cos26 



where r D is the distance between the tip of the profile 
and the nodal point. In cartesian co-ordinates, the 
corresponding equation (with reference to the same 
15 origin) is 

where x denotes axial distance and y denotes radial 
distance, as illustrated in Figure 3. This mirror 
profile is a rectangular hyperbola, whose profile 
asymptotically approaches that of a 90° cone as x and y 
20 approach infinity. 
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Example 3; Flo^a-t -jon gain - s 

The polar equation specifying the profile of a mirror 
having an elevation gain of 5 is 



z 3 

r 3 =- 



cos36 



where r 0 is the distance between the tip of the profile 
and the nodal point. in cartesian co-ordinates, the 
5 corresponding equation (with reference ' to the same 
origin) is 

where * denotes axial distance and y denotes radial 
distance, as illustrated in Figure 3. 

Exampl e 4: Elevation gain - 7 
10 The polar equation specifying the profile of a mirror 
having an elevation gain of 7 is 

< *i 

r -- ° 



cos46 



where r e is the distance between the tip of the mirror 
profile and the nodal point. In cartesian co-ordinates, 
the corresponding equation (with reference to the same 
15 origin) is 

where x denotes axial distance and y denotes radial 
distance as Illustrated in Figure 3 . 
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It should be noted that in each of the examples given 
above, it has been assumed that A-90°. If A should have 
a value other than 90°, a sharp point would be present at 
the tip of the mirror, rather than a flat surface. Such 
5 - profiles ( which are intended to be included within the 
scope of the term "dome- like" whenever this term is used 
in this specification) would be useful in excluding the 
image of the camera in the mirror, and would allow the 
best use to be made of the camera's imaging surface in 
10 capturing the environment. 

It is also within the scope of this invention to use a 
dome-like mirror which has a composite mirror profile, 
which produces different gains at different elevations 
and thus serves to magnify certain elevation bands at the 
15 expense of other elevations. ^r'^ 

As mentioned above, the circular image produced on a 
monitor screen by conventional processing of the output 
signals of a . still camera (or a video camera) having a 
charge coupled device (CCD) sensor at its image plane, 

20 when the arrangement shown in Figure 1 is used, is 
usually a warped version of the : monitored environment', in 
which the azimuth corresponds to the environmental 
azimuth and the radial distance corresponds to the 
elevation angle. An example of this warped image is 

25 shown in Figure 5, which is the image obtained with a 
video camera, using the arrangement of Figure 1, of a 
square room in which each wall has a length of 5 metres , 
and the floor-to-ceiling height is 2.5 metres. 
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To assist the interpretation of this image, it may be 
de-warped using modified image signal processing (with a 
suitably programmed computer or' microprocessor) to 
convert it into a system of cartesian co-ordinates, &,g, 
where the transformation between the warped image, 
defined on the cartesian co-ordinates (x,y), and the de- 
warped image, defined on the cartesian co-ordinates (x,(J) 
is defined by 

x=k . cos (jO 



y=k.y. sin(^) . 

Thus, the intensities at each pixel location (Jt.Q) in the 
10 de-warped image can be obtained by reading off the 
corresponding intensities at locations (x.y) in the 
warped image, as specified by the above transformation. 
Since each <*.0) pixel location in the de-warped image 
may not necessarily correspond to the centre of an (x,y) 
15 pixel location in the warped image, an Interpolation 
procedure is necessary to compute the intensity at the 
appropriate location in the warped image. The conversion 
algorithm, therefore, implements the transformation as 
well as the interpolation. 

20 The image processing algorithm for effecting the de- 
warping of the image may be any one of a number of known 
algorithms for effecting the required transformation, but 
skilled programmers will be able to generate their own 
algorithm or algorithms for this purpose. 
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A de-warped version of the image of Figure 5 is shown in 
Figure 6 . 

In some situations, it is impractical or undesirable to 
mount a camera beneath a dome-like mirror, as shown in 
5 Figure 1. In such a situation, the compact arrangement 
shown in Figure 4 may be adopted. 

In "Che arrangement shown in Figure 4, the camera 12 is 
mounted within the shell of a hollow dome-like mirror 10 
that is attached to the ceiling 11 (or other suitable 

10 structure) which forms one of the boundaries of the space 
to be monitored. A small opening at the lowermost point 
14 of the mirror 10 permits the CCD .sensor of the camera 
to receive radiation from a solid angle 9. a small, 
supplementary mirror 13 is mounted directly below the 

15 opening 14, with the reflecting surface of the mirror 13 
uppermost. Radiation from the environment being 

monitored is directed by the dome-like mirror 10 onto the 
supplementary, mirror 13, which reflects it through the 
opening 14 and onto the CCS sensor of the camera 12. 

20 Not only does the arrangement illustrated in Figure 4 
reduce the physical size of the surveillance equipment, 
but the blind spot below the central (lowermost) point' or 
tip of the mirror 10 is also reduced. The elevation gain 
of the two-mirrors combination shown in Figure 4 is 

25 distributed between the mirrors 10 and 13. 

When an electronic camera having a CCD sensor is used in 
an implementation of the present invention, the digital 
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image processing of the output signals of the video 
camera enables automatic motion sensing to be used with 
the surveillance system. With, such an arrangement, 
motion within any part of the monitored space is sensed 
automatically, and an appropriate alarm is generated. As 
noted above, this alarm may comprise colour-coding of the 
region, in the image which corresponds to the azimuth and 
elevation where motion has been sensed, and/or the 
generation of an audio alarm signal. This is a valuable 
optional feature because human observers tend to suffer 
from lapses of concentration when required to attend to 
visual tasks for extended periods. As a result, the very 
event that the surveillance system is designed to detect 
could be . overlooked. The generation of a suitable alarm 
signal means that the surveillance system can be used 
effectively without the constant supervision of an 
operator . 

The resolution of the image seen by an operator of the 
surveillance equipment is dictated by the quality of the 
imaging used in the camera of the installation. Due to 
the design of the reflective surfaces, resolution on the 
vertical axis on the user's display (elevation) is 
constant across the image, (that is, half of the number 
of lines on the CCD - the radius of the surface projected 
25 onto the cameras - represents the full range of 
elevation) . 
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The de-warping process - if used - causes resolution in 
azimuth to be directly dependant on elevation. This is 
not apparent to the operator because low elevation pixels 
in the de-warped image are stretched to ensure that the 
5 user display is rectangular. This property may at first 
appear to be a major drawback of the system. However, it 
w iH *>e apparent that nearby objects are also those at 
low angles of elevation ( they will be under the camera ) . 
Thus, despite, the fact that near objects will be sampled 
10 by fewer pixels per degree, the near objects also subtend 
a greater number of degrees. In this situation, objects 
at longer distances will be represented in the de-warped 
image by a similar number of pixels to those which 
represent objects which are close to the camera. 

15 Industrial Applicability 

The surveillance system of the present invention has 
applications in a wide number of security related areas, 
including banks, museums, art galleries, hotels and 
casinos, interview rooms in police stations, government 

20 departments , shops and shopping centres , customs , 
security apartments and carparks. The surveillance 

system is also very well suited to tasks which require 
the detection and signalling of movement in designated 
areas of a room, but not in other areas, such as the 

25 region around an oh jet d'art in an art gallery. it can 
also be used on gantries as a collision warning device. 
Another potential application is in the area of time, 
space and motion studies of a factory floor or a 
supermarket. Thus, the system of the present invention 

30 is not restricted to surveillance tasks, but is 
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applicable more generally to tasks which require 
panoramic monitoring of a large area. The present 
invention can also be extended to measure the ranges of 
objects within the panoramic field of view. in this way 
5 it can be used as a surveying instrument to map the 
topography of the surrounding environment. 

By way of summary, a brief ( and not exhaustive ) list of 
potential applications of the present invention is as 
follows: 

10 . security surveillance systems (as noted above); 

traffic monitoring and control, especially at major 
road intersections; 

airport . surveillance: panoramic monitoring of an 
airfield from a control tower; 
15 * ±n aircraft underbellies, to enable pilots to vie* 
" blind" regions beneath the passenger compartment of 
an aircraft; 

in ships and aircraft carriers, mounted on a mast to 
provide a panoramic view of decks; 

20 • as a panoramic periscope in submarines and 
elsewhere; 

as a "seeing eye" that can be mounted on a cargo 
container to warn of potential obstacles when the 
container is being moved by a crane; 
25 . remote monitoring of industrial processes ; 

monitoring of activity on workshop floors in 
conjunction with time and motion studies; and 
coastline surveillance. 
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It will be apparent from this list that although specific 
embodiments of -the present invention are illustrated and 
described in this specification, modifications of those 
embodiments may be made without departing from the 
5 present inventive concept. 
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A panoramic Paging systM for , space ^ 
—prising (i) a camera ^ { ^ } & a 

system being characterised in thatr 

(a) said mirror i«; » J-,— . ., ., 

iS a aoffi e-lil<e convex mir-or 

having £ generally sn.ootb.ly curved surf^P- 

(b) said camera is mounted relative to said mirror 
so chat a least a major portion of the £ urfaoe 
of the mirror is within the field o£ ^ q£ 

the camera; .and 

(c) said mirror has a profile such rha, th e mirror 
reflects radiation from at least a o aj or part 
of said space onto the im age plane of ' ths 
camera . 

A panoramic imaging system as defined in claim 1 tn 
-hich said dome-like mirror i s a circularly 
symmetrical mirror. 

A panoramic imaging system as defied in ol*.iu, 2 in 
which a*is of symmetry D £ said dome-like mirror is 
aligned with the optical axis of said earners. 

A panoramic imaging syst6m.es defined in claim 3 in 
which said space is a voluae, said dome-1 ik8 mirror 

15 " OUn ^ d at ° r near of said volume, and 

said camera is mounted directly below said mirror. 

A panoramic imaging system as defined in claim 3 in 

wtuch: 
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(a) s aid dome-liKe mirror is' forced a S , ho i ]0 w 
snail with an opening therein at the axis of 
symmetry of said dome-like mirror; 

(b) said camera is mounted within said shell; and 

(c) a supplementary mirror i s mounted outside said 
shell on the axis of symmetry of said dome-like 
mirror, with . the reflective surface of said 
supplementary mirror facing said dome- like 
mirror; 

whereby (i) radiation incident upon said dome-like 
mirror from at least a major part of said space is 
reflected by said dome-like . mirror onto the 
reflective surface of said supplementary mirror 
which, in turn, reflects said radiation through said 
opening and onto the image plane of said camera; and 
(ii) the aperture lens of said camera has a virtual 
position outside said shell, said virtual position 
being a distance from said opening which is a 
function of (1) the distance between the reflective 
surface of said supplementary mirror and said 
aperture lens. (2) the reflective characteristics of 
said dome-like mirror and said supplementary mirror, 
and (3) the distance between said opening and the 
reflective surface of said supplementary' mirror. 

6. A panoramic imaging system as defined in claim 5, in 
which said space is a volume and said dome- like 
mirror is mounted at or near the top of said volume 
wirh said aperture at the lowermost point of said 

dome -like mi r~r o r . 
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ceij.ii.,;. rrM ' te »°u"ted or. sal<3 

* P^orercic imasi; , 3 sysrem 

5«U + 0.5(l*«> fl J -C.r. f-o.5(l.«)j 

where rhe origin of the co-ordinate, t- „ K 
actual or - es io at the 

u * virtual positin-i , , 

said ^ ^ l ° n of the aperturo lens of 

said ca.ere. r is the di sttece of £ o . ^ ° f 

»irror profile from the 0r - * °* 6 ° n *^ 

■inclined ar M ^ e ° * * d± ~ c «°« 

of the camera „ , h . „ , t0 the °Pti=al axis 

OSest to the aperture lens of r ho 

" s or the camera- 

A panoramic i maginff sy 

pr^cedinc cla i„. in „ id ; lned " any 

S l e c. tonl c c „ Ka ««» i= „ 

<CCD, sensor „ ' ° = ha -° e couple a « vlce 

screen. space °n a monitor 
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A panoramic in laging systgffi £s 

which said i^aae on *.x " Cla:UD 9 < in 

o^age on the monitor screen < s * 

i^age of said space a wa rped 



A panoramic Paging syste* as dftf n 

which said signal proce ssing ffieans . Claun 9 ' 

produce a de . wanW( , < 9 ^ S ±S P r °Sramn,ed to 

a de^arped i m£ge of said Spsce 
monitor screen p ce on S3id 
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cj.aiin io o r claim n ±n „ h , . 
Processing me£ns ^ clude / a ^ 

which is adant^ sea.s^ng slgorithn, 

adapted to sense motion within 

to provide an Indication of rt 
said motion * °° CUnrer,ce °* 
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v,hich said indication comprises th. ^ • , ' 

^ region in said space . d ** play ° f 

^ spa.ce within wH«' r*H m,-^,- ~ 

sen sed on said „cu tor screen ^ m ™- e " h »* 

f nr -a-rj * our ojr co]. ours used 

OI Said °n said monitor screen. 

IS- A panoramic imaging syst;em as tfa „ tnod . 

claims 9 to u w ed ln en V one of 

*° 14, Ln which said - ano r, • 

oanera -am era Js a video 
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reference to the accompanying dra„ ings * 



\tfO 95/06303 



1/3 



PCT/AU94/00501 




Fig. 2 




'Fig. 6 



